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(57) Abstract 



A document-centered user interface aichitecture for a computer system employs parts as the fundamental building blocks of all 
documents. AD data is stored in the system as a part, which is comprised of contents and an associated editor. The conteots and the 
functionality of the editor are available to the user wherever the part is located, whedier in a document, on a desktop or in a folder. Parts 
function as containers for other parts, thereby facilitating the compilation and editing of multimedia or compound documents. When material 
from one part is inserted into another part, the onnputer system automatically determines whether the added material is incorporated into 
the intrinsic contents of the receiving part or embedded as a separately identifiable part 
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SYSTEM FOR AUTOMATICALLY DETERMINING THE 
STATUS OF CONTENTS ADDED TO A DOCUMENT 

Field of the Invention 

The preset invention is directed to the interface between a 
5 computer and the human user of that computer, and more 
particularly to a computer-human interface architecture that 
determines whether mat^al added to a document should be 
incorporated as part of the intrinsic contents of the document or 
embedded in the documoit as a separately identifiable object. 

10 Background of the Invention 

Currently, many applications for use in personal computers 
are written as single monolithic programs^ each of which is 
designed with a specific primary function to be performed. For 
example, word processing programs are designed to create and edit 

IS textual documents. Although not designed for text processing, 
many other types of programs also provide for incidental use of 
text. For example, spreadsheet programs enable text to be inserted 
in the spreadsheet as descriptors for rows and columns of 
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information. Similarly, graphics applications such as painting and 
drawing programs provide for the use of text as labels within the 
paintings and drawings. While all of these different types of 
programs provide text editing facilities, they differ in the specific 

5 manner in which text is edited. Some of the applications provide a 
large number of capabilities for this function, while others are quite 
limited. Consequently, users are required to learn a different way 
to perform the same task, i.e. edit text, for each of the different 
types of applications. 

10 It is desirable, therefore, to provide an architecture in which 

a component of a document, such as text, can be employed 
wherever needed while at the same time be created and edited in a 
consistent manner, regardless of the particular environment in 
which it resides. With such an approach, the same set of 

IS commands can be used to edit text whether it is present in a word 
processing document, a spreadsheet, a graphic document or any 
other docum^t. The user only has to learn about one text editor, 
and can thereby become more comfortable with the editing process. 
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Further along these lines, it is desirable to be able to edit 
the components of a compound document within the document 
itself. In the context of the present invention, a compound 
document is one in which two or more different types of programs 
S are used to produce the sq>pearance of a single unified piece of 
information. For example, it could be a text document which 
includes a drawing created by a graphics program. These two 
types of information have a containment relationship to one 
another. More particularly, the text document functions as a 

10 container for the graphic object, such that the graphic object moves 
with and is affected by the text (container) document* 

In the past, a compound document of this type would be 
created by running two separate applications. A word processing 
program was used to generate the textual portion of the document. 

IS A separate graphics application was used to generate the drawing. 
To incorporate the work products of these two programs into a 
single document, the drawing would be stored in a particular data 
type or format, and the contents of this data structure would be 
placed in the word processing document, for example by using a 

20 cut and paste operation. Often, the merging of these separate 
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pieces of information into a single document was accomplished 
through a third application program, referred to as a presentation 
managtir. 

Once the drawing was embedded in the word processing 
5 document, its contents became static. If the drawing needed to be 
changed, it was necessary for the user to call up the graphics 
program, retrieve the original drawing, and edit that drawing within 
that program. The editing took place outside of the compound 
document, and the user could not view the changes in context as 
10 they were being made. The revised drawing would then have to be 
embedded in the word processing document, in place of the original 
drawing. 

When a drawing, or other content, was embedded in another 
document, it existed as a separate object, distinct from the intrinsic 

IS contents of the document itself. In cases where die added material 
is of die same type as the document's contents, however, the user 
prefers that the added contents be incorporated into the existing 
contents of the document, rather than embedded as a separate 
object. While some programs allowed the user the option of 

20 incorporating material into the contents of a document, this result 
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could only be accomplished through the use of intervening dialog 
boxes. For example, if material was added by means of a cut-and- 
paste or copy-and-paste operation, the user had to decide the 
method of pasting (embed or incorporate) each time added content 
S was pasted into a document • - 

To overcome the need to switch between various 
applications, it is desirable to provide an approach which enables 
the various components of a compound document to be edited 
within the document itself. Not only does this approach save time 

10 and simplify the procedure, it enables the user to si^ the changes in 
context as they are being made. Furthermore, when material is 
added to a document, it is preferable to have that matmal be 
automatically incorporated into the intrinsic contents of the 
document, whatever possible. 

IS Accordingly, it is an objective of the present invention to 

provide a system in which a consistent user interface can be 
provided for a task, regardless of the environment in which the task 
is to be performed. In addition, it is the objective of the present 
invention to eliminate the need for user input to determine whether 
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material that is added to a document should be incorporated or 
embedded in the document. 

Brief Statement of the Invention 

In pursuit of these objectives, the present invention provides 
5 a uniform interface that enables all different kinds of information- 
conveying elements to be combined and manipulated within 
documents in ways that were not previously possible. In the 
context of the present invention, a document is considered to be the 
element of an interface whose purpose is to present information to 

10 the user and allow that information to be edited, for example a text 
document or a graphics document. 

The uniformity of the interface is provided through a 
document-centered architecture which employs a fundamental 
building block, hereinafter referred to as a "part**, to accomplish 

IS tasks. A part is a self*contained element that is comprised of two 
primary components, its content and a manipulator for that cont^t. 
These two components are always available to the user, regardless 
of where the part may reside within the system, e.g. as an element 
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of a document, in a folder, or on a desktop. The manipulator for 
the part can be any type of editor or viewer for the contents of the 
part, and it can be selected by the user, if necessary. For example, 
if the contents of a part is text, the editor which is a component of 
5 that part can be any word processor with which the user has 
familiarity. That word processor can be used to edit the textual 
contents of the part, wherever they may be embedded. Thus, even 
if the part is embedded, within a nontextual document, e.g. a 
spreadsheet, the textual contents of the part can still be edited with 

10 the user-selected word processor. 

An interface which exhibits this feature has a significant 
inherent characteristic: it is simultaneously simpler and more 
powerful. The simplicity results from the fact that the user only 
has to learn one way to perform a particular task, such as editing 

IS text. At the same time, it is more powerful because the text editor 
can be a fully functional module, which replaces less capable ones 
currently found in many types of applications. 

The two basic components of a part enable it to function as 
both an object (due to its inherent contents) and as an application 
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(resulting from its inherent manipulator or editor). Due to these 
characteristics, a part can provide a number of different functions. 
For example, every document is a part, and every part can act as a 
document. To illustrate, a text part is a document containing 

5 characters, and a graphic part is a document containing lines and 
circles. In the present invention, folders, and even the desktop, 
constitute parts, and therefore can contain contents that were 
previously associated with documents only, such as text. This 
feature of a part provides a uniformity that does not exist in 

10 currently available applications, where objects such as folders 
cannot function as documents. 

As a further feature, the present invention automatically 
controls whether material added to a document is incorporated as 
part of the document's intrinsic contents or embedded in the 

IS document as a distinct object. All parts are identified as belonging 
to a particular category of information, such as text, graphics, 
video, etc. Within each category, there may be different data 
formats or part types When material from one part is to be added 
to another part, the respective categories of the two parts are 

20 compared. If they are the same, the added material is automatically 
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incorporated into, and becomes a portion of, the intrinsic contents 
of the receiving part. If the two parts (donor and destination) 
belong to the same category but have different data formats, the 
format of the added material is converted into that of the destination 

S part, and then incorporated into its contents. 

When the donor and destination parts belong to different 
categories, a new part is created with the added material as its 
contents. This new part is then embedded in the destination part, 
i.e. the destination part functions as the new part's container. 

10 Further features of the invention are explained hereinafter 

with reference to illustrative examples shown in the accompanying 
drawings. 



Brief Description of the Drawings 

Figure 1 is a block diagram of the main components of a 
15 computer system; 

Figures 2A and 2B are two views of a screen display, 
illustrating the dragging of a part from a folder to a document; 

Figures 3A and 3B are two views of a screen display, 
illustrating the activation of frames; 



wo 94/27228 



PCTAJS94/05027 



10 

Figure 4 is an illustration of a selected frame within an 
active frame; 

Figure SA and SB are diagrams illustrating two examples of 
frame structures, and Figures 5C-5F are flow diagrams for 
5 examples of negotiations which take place between an embedded 
part and a containing part with respect to frame attributes; 

Figure 6 is a flow chart of the process for launching an 

editor; 

Figures 7A-7E are a sequence of screen displays illustrating 
10 the dragging of a part between a desktop and a document; 

Figures 8A and 8B are a flow chart for the process of 
dragging and dropping a part; 

Figures 9A-9C, lOA-lOC and IIA-IIC are respective 
sequences of screen displays illustrating the copying of material 
15 from one part into another part; 

Figure 12 is a flow chart of the process for placing material 
from one part into another part; 

Figure 13 is a screen display illustrating a part having 
restricted access; and 
20 Figure 14 is a screen display illustrating a parts bin. 
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Detailed Description 

To facilitate an understanding of the present invention and 
the advantages offered thereby, features of the invention are 
described and illustrated with reference to the use of the invention 
5 in a Macintosh® brand computer supplied by Apple Computer, Inc. 
It will be appreciated by those having familiarity with the relevant 
technology, however, that the principles of the invention are not 
limited to this particular type of operating environment. Rather, 
the concq)ts which underlie the invention are applicable to any type 

10 of computer operating system in which it is desirable to provide a 
uniform interface for the compilation and editing of documents, 
particularly compound documents. 

A typical computer system, of the type in which the present 
invention can be employed, is illustrated in block diagram form in 

15 Figure 1. The structure of the computer itself does not form part 
of the present invention. It is briefly described here for subsequent 
understanding of the manner in which the features of the invention 
cooperate with the structure of the computer. Referring to Figure 
1, the system includes a computer 10 having a variety of external 

20 peripheral devices 12 connected thereto. The computer 10 includes 
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a central processing unit 14, a main memory which is typically 
implemented in the form of a random access memory 16, a static 
memory that can comprise a read only memory 18, and a 
permanent storage device, such as a magnetic or optical disk 20. 
5 The CPU 14 communicates with each of these forms of memory 
through an internal bus 22. The peripheral devices 12 include a 
data entry device such as a keyboard 24, and a pointing or cursor 
control device 26 such as a mouse, trackball or the like. A display 
device 28, such as a CRT monitor or an LCD screen, provides a 

10 visual di^lay of the information that is being processed within the 
computer, for example the contents of a document. A hard copy of 
this information can be provided through a printer 30, or similar 
such device. Each of these external peripheral devices 
communicates with the CPU 14 by means of one or more 

15 input/output ports 32 on the computer. 

The present invention is particularly directed to a system 
which controls the manner in which the CPU 14 handles 
information that is manipulated by the keyboard 24 and the cursor 
control device 26, as well as the manner in which that manipulated 

20 information is portrayed to the user through the display device 30. 
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An example of an interface which employs the principles of the 
present invention is illustrated in Figures 2A and 2B. These figures 
iUustrate a desktop 34 which defmes a worlcspace 36. Also 
included on the desktop is a menu bar 38. Within the workspace 
5 36 are two windows, 40 and 42. The left-hand window 40 is a 
folder window which contains icons representing various parts. As 
shown in Figure 2A, three icons 44 pertain to text parts, a fourth 
icon 43 represent a graphic part and another icon 46 represents a 
spreadsheet part. A sixth icon ^9 is a stationery icon representing 

10 a part that can be used for the creation of presentation slides. In 
Figure 2A the right-hand window 42 contains a text document. 
More particularly, the document includes a text part 50. This text 
part can be created by typing text within the window 42, or by 
opening in the window a previously-created text part that was 

IS rq)resented by an icon. 

Figure 2B illustrates the result of an operation in which the 
graphic icon 45 has been dragged from the folder in the left 
window 40 to the text document in the right window 42. As is 
known in interfaces which provide a desktop metaphor, such as that 

20 shown in Figures 2A and 2B, dragging is an operation in which 
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objects can be moved or copied on the desktop and within windows 
through the actuation of the cursor control device 26. Once the 
graphic icon is placed within the document, its contents are 
displayed in a frame 52. Since the document now contains 
graphic element as well as a text element, it is referred to as ; 
compound docunient. 

The text 50 and the graphic element within the frame 52 
pertain to separate respective parts which together make up the 
document. As described previously, a part is a self-contained entity 
which is comprised of content and a manipulator for that content. 
These two components are always available when the part is 
accessed, regardless of where it is located on the desktop. n,us, 
when the graphic icon 45 is located in the folder of window 40, as 
in Figure 2A, it represents a part comprising a graphic element and 
an editor for that element. When the part is moved to the 
documem in the window 42, its contents go with it and the 
functionality of an editor for those contents is available to the user. 

The significance of this feature is the fact that the contents 
of a part can always be edited, or otherwise manipulated, wherever 
they are located. Thus, in the compound document of Figure 2B, 
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the textual content SO can be edited within the document, for 
example by using the same word processor that was used to 
originally create the text. Since the graphic editor is present as a 
component of the part represented by the frame 52, it enables the 
5 graphical contents of the part to be directly edited in place. Thus, 
unlike prior applications for personal computers, it is not necessary 
for the user to open a graphics application, for example in another 
window on the desktop 34, edit the graphical information as 
desired, and then export the edited graphics into the document in 

10 the window 42, Rather, the user can directly edit the contents of 
the frame 52 within the context of the document in the window 42, 
without having to implement any intervening steps. The user can 
focus on document content, rather than an application program, and 
take advantage of the context provided by the surrounding 

IS document. 

A part, therefore, is a self-contained object which is 
autonomous of the underlying system technology. It can be used in 
any type of software engine or environment, and the user does not 
have to learn the operation of a particular editor for the part. As 

20 long as an editor for that category of part is represented in the 
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underlying system, the user can take full advantage of the part's 
capabilities. Furthermore, if a text part was created with a 
particular text editor (word processor), the user is not constrained 
to use only that text editor with the part. If the user has more 
5 familiarity with the commands of a different word processor, that 
processor can be designated as the preferred editor for all text parts 
in the computer system. 

Parts are not limited to documents and their contents. 
Rather, every object in the system comprises a part. Thus, the 

10 folder in the window 40 is a part whose contents are the icons 44- 
48. Similarly, the desktop 34 is itself a part, as well as dialog 
boxes and other user interface objects. 

Frames and icons are alternative representations of a part. 
Users can toggle between the frame and icon representations of a 

IS part by means of a suitable keyboard or menu command. If a 

document is large, e.g. it consists of several pages, only a portion 
of it is visible in the frame. Therefore, frames, like icons, can be 
opened into windows, to thereby allow all of the contents of large 
parts to be seen, through scrolling for example, and edited using 

20 the functions available with a window, such as magnification. 
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Similarly, they can be closed from a window back into a frame. A 
frame differs from an icon, in that it allows its contents to be edited 
in place. A frame is also different from a window because a 
frame, or even multiple frames, can exist inside a window. A 
5 window is a transitory view of objects, which remains open only 
while a part is being edited or examined. In contrast, a frame is a 
permanent representation of a part's contents. 

A part, whether it is in the form of an icon or a frame, can 
act as a container for other parts. In the example of Figure 2B, the 

10 text SO is located within a frame whose border is the same as that 
of the window 42. This larger frame contains the smaller frame 52 
for the graphic part. Since the text part is the outermost part in the 
window, it is labelled as the **root part". Generally speaking, the 
root part establishes the basic editing behavior for a window. 

15 A root part can be generated by means of a stationery icon, 

such as the icon 48 depicted in Figure 2A. In general, a stationery 
icon represents a predefined part and functions as a metaphor for a 
pad of paper in the physical world. When it is desired to generate 
a new document, a person tears a blank sheet of paper from the pad 

20 and places the contents of the document, e.g. words, drawings, 
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etc., on it. The stationery icon works in the same manner. When 
the user desires to create a new text document, a text stationery 
icon is opened, for example by double-clicking on it. In response, 
the stationery icon creates (or "tears ofP) a copy of itself, which is 
5 opened into a window or a frame. This copy is an ordinary part 
which can be blank, i.e. it has no contents initially, or it could have 
certain pre-set information, such as letterhead or a company logo. 
The user then enters additional content, e.g. characters, into the 
document. 

10 The stationery itself is not opened as a window or a frame; 

it only exists as an icon. Only the tom-off copy is opened. Thus, 
when a stationery icon is dragged into a document, rather than 
opening itself into a frame, a copy is "torn ofr, placed in the 
document, and that copy is opened into a frame. The stationery 

IS icon itself returns to the position it occupied before the move. This 
prevents the stationery from being accidentally inserted into a 
document, and possibly lost. 

Since a frame can function as a container, it exhibits some 
of the known properties of a window. For example, when a user 

20 selects a part contained within a frame, such as a graphical object, 
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the frame containing that part becomes active* When a part is 
active, it receives commands and keyboard events, and its menu 
and other user interface options are displayed. An example of the 
activation of a frame is illustrated in Figures 3A and 3B. Figure 

5 3A illustrates a desktop having a window 54 which includes a 
compound document containing text and a graphic part. In this 
compound document, the text forms the root part of the document. 
In the example of Figure 3A, a portion 56 of the text has been 
selected, and is represented by the reverse video display. In this 

10 case, the frame which contains the selected part is contiguous with 
the border of the window itself. The menu bar 58 for the desktop 
includes those commands which are appropriate for editing the text. 

Figure 3B illustrates an example in which a graphical 
component, i.e. the triangle 60, has been selected. The frame 

15 which contains this graphical component is now active, and its 

border is indicated by the dotted line 62. Since the active frame is 
now a graphic part, the commands in the menu bar 58 have 
changed to those which are appropriate for editing graphical 
content. In addition, a palette 64 is displayed on the desktop, to 
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provide those tools which are necessary for editing the graphical 
contents of the active frame. 

In addition to activating frames, a user may also select 
them. The selection of a frame can be accomplished, for example, 
5 by activating a frame so that its border is visible, and then pressing 
a button on the cursor control device while the pointer of the cursor 
is positioned over the border of the frame. Figure 4 illustrates a 
frame 66 that contains both graphical and textual content. The 
border 68 of the frame is indicated by a dotted line, to denote the 

10 fact that it is an active frame. Within this active frame is a frame 
70 containing the textual content. This frame has been selected by 
depressing the button of the cursor control device, while the pointer 
of the cursor 72 is positioned over the border 73 of the frame. To 
denote the feet that the frame 72 has been selected, the border 73 

15 of the selected frame 72 has a different appearance from that of the 
active frame 66. For example, the border might be wider as shown 
in Figure 4. In addition, it can be provided with resize handles 74, 
which enable the size and shape of the frame to be changed in a 
known manner. 
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In general, whenever a part or some of the contents of. a 
part is selected, the smallest frame containing the selected element 
becomes the active frame. Thus, if a word within the frame 70 is 
selected, for example by clicking a cursor control button while the 
5 cursor is positioned over the word, the frame 70 changes state from 
selected to active. At the same time, the border 68 of the frame 66 
disappears from view since this frame is neither active nor selected. 
Conversely, if the frame 66 is selected by clicking on its border 68, 
the border 73 disappears, the border 68 changes appearance, and 

10 the border of the frame (or window) containing the frame 66 (not 
shown in Figure 4) is highlighted to show that it is active. 

In the examples of Figures 3A, 3B and 4, the frames are 
illustrated as having a rectangular shape. While this shape is 
preferable for displaying the contents of most types of parts, the 

IS shape of the frame need not be limited to being rectangular. 

Rather, the frame can have any shape which suits the display of its 
contents. 

The frame is not an element of the part itself. Rather, a 
frame is a separate data structure which functions as an interface 
20 between the container part and the embedded part. Each frame has 
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a number of characteristic attributes, which are denoted as various 
"shapes". Each of these shapes represents a geometric description 
of an area that defines certain document context that is shared 
between the container part and the embedded part. Some of the 
5 shapes are controlled by the container part, whereas others are 

determined by the embedded part. In general, these various shapes 
can represent any area of the coordinate space of the document 
context. In this regard, the shapes of a frame can represent disjoint 
areas as well as contiguous areas. 

10 The various characteristic shapes of a frame are described 

with reference to Figures 5A and 5B. In these figures, the left- 
hand illustration depicts a compound document containing text (the 
root part) and a graphic part, represented by a triangle. The right- 
hand illustration shows the shape attributes of the frame for the 

15 graphic part. 

One of the characteristic attributes of a frame is the Frame 
Shape. This shape is determined by the container part, and 
describes to the embedded part the geometric area that it should use 
to lay itself out. Referring to Figures 5A and 5B, the Frame Shape 

20 is depicted by the outermost solid border 75. The area defined by 
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this shape might affect the scaling, line breaks, or other similar 
layout decisions that must be made by the editor for the embedded 
part. Only the container part can directly change the Frame Shape. 
When this shape is changed by the container part, a notification is 

5 provided to the embedded part, to enable appropriate changes to be 
made in the content of the embedded part. 

A second characteristic attribute of a frame is its Clip 
Shape, which is depicted by the shaded border 76 in Figures 5 A 
and 5B. The Clip Shape describes to the embedded part the 

10 geometric area in which the contents of the part can be drawn. In 
the example of Figure 5A, the Clip Shape is coextensive with the 
Frame Shape. However, in the example of Figure 5B, the lower 
right-hand portion of the triangular graphic part is overlaid by a 
table. Therefore, the Clip Shape 76 for the graphic part excludes 

15 this area which is to be overlaid by the table. The Clip Shape is 

determined by the container, i.e. the text document which forms the 
root part of the example illustrated in Figures 5 A and SB. Only the 
container can change the Clip Shape, and when it does so it notifies 
the embedded part of the change. 



wo 94/27228 



PCT/US94/05027 



24 

Another characteristic attribute of a frame is the Used 
Shape, depicted by the solid line 77 in Figures 5A and 5B. This 
shape is determined by the embedded part, and describes to the 
container the part of the geometric area that is actually being used 
5 by the embedded part. This shape is necessarily a subset of the 
Frame Shape. The container part is not allowed to draw inside the 
Used Shape, but is free to draw in regions outside the Used Shape, 
even if they are within the area of the Clip Shape and the Frame 
Shape. The embedded part can change the Used Shape, and when 

10 it does so it notifies the container part of the change. 

A fourth shape which is a characteristic attribute of a frame 
is its Active Shape, which is depicted by the shaded line 78 in 
Figures 5A and 5B. This shape is determined by the embedded 
part, and describes to the container part the geometric area in 

15 which the embedded part should receive geometrically related 
events such as cursor control input. Whenever an event occurs 
within this area, the container part must pass the event to the 
embedded part, rather than respond to the event itself. However, 
the container part is free to respond to events which occur in 

20 regions outside the Active Shape of the embedded part. 
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Another attribute of a frame is its group tag. The frame 
group tag is an identifier, such as a number, which uniquely 
identifies a set of related frames in a given container. The 
container part uses the frame group tag to control the geometric 
5 relationship of the various frames in the group. 

The frame allows container parts and embedded parts to 
negotiate document structure with minimal knowledge of one 
another's internal constraints. Examples of negotiations that can 
take place between the containing part and the embedded part are 

10 illustrated with reference to Figures 5C-5F. In the example of 

Figure 5C, the Frame Shape is being altered by the container. This 
procedure may be initiated by a command from the user to change 
the size of the frame, for example. At Step 501, the containing 
part determines a new Frame Shape for the embedded firame, and 

15 notifies the flame of this new shape. In response thereto, at Step 
502, the embedded frame transforms the shape from the containing 
frame's coordinate space into that of the embedded frame. This 
new shape is stored as the Frame Shape, and passed on to the 
embedded part. At Step 503, the embedded part adjusts itself to 

20 the new Frame Shape. For example, a text editor may reset 
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margins for the layout of the text. If the Used and Active Shapes 
are different from those of the Frame Shape, the embedded part 
also computes these new shapes. Control is then returned to the 
containing part, and at Step 504 the containing part changes the 
5 Clip Shape of the frame to correspond to the new Frame Shape, 
and notifies the frame of the new Clip Shape. 

Figure 5D illustrates the procedure that takes place when the 
embedded part requests a new Frame Shape. This request may be 
brought about as a result of an editing of the embedded part's 

10 contents, which necessitates additional space for the contents to be 
properly laid out. At Step 505, a determination is made that the 
embedded part needs a new Frame Shape, and a request is sent to 
its display frame for a new shape. The old shape can be stored by 
the embedded part, in case it will be later needed. At Step 506, the 

15 display frame transforms the shape from the embedded frame's 
coordinate space into that of the containing frame, and passes the 
request onto the containing part. At Step 507, the contdning part 
determines how to respond to the request. It may grant the request 
for the desired shape, or determine that a smaller shape is required. 

20 Once the Frame Shape is determined, the Clip Shape for the frame 
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is also modified, and a notification is provided to the frame as to 
the allowed shape. At Step 508» the display frame transforms the 
sh^ from the containing frame's coordinate shape into that of the 
embedded frame. The new shape is stored as its Frame Shape, 
5 which is passed on to the embedded part. At Step 509, the 

embedded part changes its display frame's Used and Active Shapes, 
to adjust to its new Frame Shape. 

Figure SE depicts a procedure in which the container part 
changes the Clip Shape. Referring to Figure 5B, for example, the 

10 need to change the Clip Shape may result from an action by the 
user, in which a portion of the area encompassed by the display 
frame is to be overlaid with the contents of a different part. At 
Step 510, the containing part determines a new Clip Shape for its 
display frame, and notifies the frame of this new shape. At Step 

15 511, the display frame passes the notification on to the embedded 
part. In response thereto, at Step 512, the embedded part adjusts 
itself to the new Clip Shape by changing the manner in which its 
content is laid out, as appropriate. 

Figure 5F depicts a procedure in which the embedded part 

20 changes its Used Shape. Again, the need for such a change may 
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arise from an editing of the contents of the part by the user. At 
Step 513, the embedded part picks a new Used Shape for its display 
frame, and notifies the frame of this new shape. At Step 514, the 
display frame passes the notification on to the containing part. In 
5 response thereto, at Step 515, the containing part adjusts itself to 
the new Used Shape. For example, it may change the way its 
content is laid out to flow differently around the new shape. 

The frame, therefore, is a data structure which negotiates 
the required space allocation between an embedded part and a 

10 containing part. Through the use of various shape attributes, the 
frame enables the containing part and the embedded part to utilize 
the available space in an efficient manner, without requiring that 
each part have detailed information as to the contents or behavior 
of the other part. Thus, compound documents can be assembled 

15 with a great degree of flexibility. 

The editor, or other manipulator, for a part is analogous to 
an application program in a conventional computer system. It is a 
software component which provides the necessary functionality to 
display a part's contents and, where appropriate, present a user 

20 interface for modifying those contents. It may include menus. 
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controls, tool palettes and other interaction techniques. For a part 
which is capable of containing other parts, its editor takes these 
contained parts into consideration when displaying its content. For 
example, a text editor may wrap its part's contents so that it does 
5 not appear in the Used Shape of an embedded graphic part. 

In order to use the functions provided by a conventional 
application, the program must be launched or booted. In other 
words, it must be stored in the system's active memory, e.g. the 
RAM 16. In a similar manner, the editor for a part must be 

10 running in the system's memory or otherwise accessible to the 
CPU. To do so, the editor must be installed on the system. For 
example, it can be stored in a folder on a hard disk 20, and called 
when an associated part is accessed. To conserve system 
resources, all installed editors may not be loaded into the system 

15 memory at all times. Rather, they can be launched as needed. For 
example, an editor for a part can be launched when the part is 
dragged into a document, since this is the time that the editing 
capabilities are most likely to be needed. 

The process for launching an editor, or other manipulator, is 

20 depicted in the flowchart of Figure 6. Referring thereto, at Step 
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601 the system looks for a part to be activated. The accessing of 
this step may be in response to an event flag that is generated 
whenever the mouse button is depressed, or it could be a step in a 
script. Once the activation of a part is detected, the system 
identifies the editor associated with that part (Step 602). In this 
regard, every part has certain properties associated with it. One of 
these is a pointer which points to the editor for that part. Two 
other properties are the part's category, e.g. whether it is text, 
graphic, spreadsheet, etc, and its type, which is an indicator of the 
formal for the part's contents. For example, one text part could be 
of the "Word Processor A" type, and another's type could be 
"Word Processor B", although both belong to the same category 
(text). Often, a part's type will be determined by the last editor 
used on the part's contents. Generally speaking, a category is a set 
of part types, and determines the set of part editors or viewers that 
are applicable to a given part. The properties of a part are stored 
along with its contents. For example, all of a part's properties can 
be contained in a "property sheet" that is stored with the part's 
contents. 
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Once the editor (or part type) has been identified at Step 
602, the system determines whether that editor is already running 
in memory (Step 603). If so, the system returns to the main 
routine and awaits further user action. If the editor is not already 
5 running, the system checks whether the editor identified in Step 602 
is installed on the system (Step 604). If the editor identified at Step 
602 is installed on the system, that editor is launched at Step 605, 
so that it can be used to edit the part's contents. 

It may be the case, however, that the part's editor is not 
10 currently installed on the system. In this case, the decision at Step 
604 would be negative, and in response thereto the system 
determines whether the user has selected a preferred editor for that 
category of part (Step 606). More particularly, a user can select a 
preferred, or default, editor for each different category of part. 
15 Thus, if the user is most familiar with the function commands of a 
particular word processor, that processor can be selected as the 
preferred text editor for all parts in the text category. If a 
preferred editor has been selected, the system checks whether the 
preferred editor is installed (Step 607), and if so.it is launched at 
20 Step 608. 
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If no preferred editor has been specified, or if the preferred 
editor does not exist on the system, a check is made at Step 609 
whether any editor for that category of part is present on the 
system. If not, an appropriate warning is provided to the user 
(Step 610), and the system then returns to the background routine. 
If one or more editors for that category of part is available, they 
are listed and the user is prompted to select one (Step 61 !)• In 
response to a selection by the user, the selected editor is launched 
(Stq) 612) and it is stored as the preferred editor for that category 
of part (Step 613). The system then returns to the background 
routine to await further user input. 

The user can be provided with the option of changing the 
designated editor for a part at any time, for example through an 
appropriate menu command. By invoking this command, the user 
causes the pointer that is stored as one of the part's properties to be 
changed to the location of a different editor installed on the system. 
Menu commands can also be provided to enable the user to select 
prefored editors at different levels, for example at the document 
level (all text parts in a document have the same editor) and at a 
global (system-wide) level. Whenever the user accesses the menu 
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to select or change an editor for a part, only the set of installed 
editors which are appropriate for that category of part are displayed 
to the user for selection. 

If desired, the system can keep a record of the last time that 
each editor running in memory has been used. If an editor has not 
been used for a predetermined length of time, the system can 
automatically delete the editor from memory, to thereby conserve 
system resources. Also, when a document is closed, the editors 
associated with the document's parts can also be closed, unk^^ they 
also pertain to other parts which are currently open, for exar - je in 
another document. 

Because of the autonomous nature of parts, the containers 
for various parts are no longer limited to specific types. For 
example, in the past a text document could only contain graphics or 
other content which was in specific data formats. Limitations of 
this type are removed with the architecture of the present invention. 
Basically, any part appears as a "black box" to any other part. A 
containing part need not have any information about the internal 
structure or semantics of other parts contained within it. The 
containing part merely functions as a wrapper for the other parts 
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contained in it. This type of architecture greatly facilitates the. 
compilation and editing of compound documents. 

To further illustrate, a document can be represented by an 
icon which is located on the desktop itself. Referring to Figure 
7A, the illustrated desktop includes a window 79 containing a text 
document. Also located on the desktop is an icon 80 for a graphic 
document. Each of the window and the icon rq)resents a respective 
part. In the case of the text document located within the window, 
the part is represented in the form of a frame, whose borders are 
contiguous with that of the window itself. In accordance with the 
present invention, the part represented by the icon can be directly 
manipulated by the user and placed within the text document, 
without the need for intervening dialog boxes or the like. This 
operation is described hereinafter with reference to the illustrations 
of Figures 7A-7E and the flowchart of Figures 8A and SB. 

To place the graphical document rq)resented by the icon 80 
into the body of the text document within the window 79, the user 
first positions the mouse pointer over the icon (Step 801) and 
depresses the mouse button to select the icon (Step 802). In 
response thereto, the system highlights the display of the icon, to 
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indicate the fact that it is selected (Step 803). In the example of 
Figure 7A, the highlighting of the icon 80 is indicated by the 
reverse video representation of the icon's picture and its title. 

While holding the mouse button down, the user moves the 
cursor to the desired location for the graphic part (Step 804), As 
iUustrated in Figure 7B, the selected icon follows the movement of 
the cursor. The movement of the icon with the cursor is known as 
a "drag" operation. While the icon is being dragged to the desired 
location, a ghost outiine 81 of the icon remains on the desktop, to 
indicate its original position. Alternatively, the ghost outline can 
move with the cursor, and the full icon rq)resentation remain in the 
original position, as depicted in Step 805. 

During the dragging of the icon, the system detwmines 
whether the cursor raters a window (Step 806). As the icon 80 is 
positioned inside the window 79, the window itself becomes 
highlighted. If the system detects that the dragged icon enters a 
frame within the window (Step 807), the frame is apprt^riately 
highlighted, to indicate that it can receive the dragged object (Step 
808). If the icon continues to be dragged so that it goes back 
outside of the frame, the highlighting on the frame is removed 
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(Step 809). Conversely, if plural frames are nested inside one 
another, each successively nested frame becomes the active one as 
the icon is dragged inside it. In this manner, the system provides 
feedback regarding the frame that will be active and the location at 
which the icon will be dropped if the cursor control button is 
released. 

Once the dragged icon is at the desired location, the user 
releases the mouse button (Step 810). At this point, the system 
determines the preferred representation for the part, i.e. whether it 
should appear as an icon or as a frame (Step 811). In the example 
of Figures 7A-7E, since the part has been placed in a document, its 
preferred representation is as a frame. Accordingly, as shown in 
Figure 7C, the graphic part is displayed as a frame 86, with its 
upper left comer corresponding to the position of the cursor when 
the mouse button was released (Step 812). As also shown in 
Figure 7C, the frame is illustrated with a thicker border and resize 
handles, to indicate the fact that it is the selected part. 

The reverse operation can be performed to move the part 
from the document back onto the desktop. Furthermore, it is 
possible to copy the part, so that its contents remain within the 
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document while another representation of the part is placed on the 
desktop. To do so, the user depresses a function key, such as a 
control or option key on the keyboard, while dragging the frame. 
The simultaneous depression of the function key along with the 
actuation of the mouse button provides an indication to the system 
that a copy operation is to be performed. Then, while continuing 
to hold the mouse button down, the user drags the frame to the 
desktop, as illustrated in Figure 7D. Note that the contents 88 of 
the graphical part remain within the text document, and a second 
copy of the part appears in the frame 86. Once the ftame is moved 
to the desired location on the desktop, the mouse button is released, 
and the system again determines the preferred representation for the 
part. In this case, since the part is located on the desktop, the 
preferred representation is as an icon. Thus, an icon representation 
80 of the part is displayed on the desktop, as shown in Figure 7E. 

Since the part is directly manipulated between the desktop 
and the document in the system of the present invention, the 
metaphor of the desktop is better preserved. In other words, the 
user picks up a desired part from the desktop and places it directly 
in the document, without any intervening operation such as filling 
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out dialog boxes or the like. As a result, the amount of time and 
number of steps that are required to introduce new content into a 
document is reduced. 

Further details regarding one manner in which the system 
can implement the dragging and dropping of objects between a 
desktop and a document are described in Application Serial No. 

. , filed on March 3, 1993 by Robert G. Johnston, 

Jr., Mark L. Stem and David L. Evans, entitled "Method and 
Apparatus For Improved Manipulation of Data Between an 
Application Program and the File System on a Computer Controlled 
Display System", the disclosure of which is incorporated herein by 
reference. 

As described above, each part has a property which is 
referred to as its preferred representation. This property 
determines whether parts contained inside a given part are normally 
displayed as icons or as frames, in order to obtain a behavior which 
is traditionally associated with both folder and document windows. 
Generally speaking, all parts are classified as either a desktop part 
or a document part. A desktop part is one whose purpose is to 
contain or operate on other parts, such as a folder, printer or 
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mailbc - The preferred representation for a part contained in a 
desktop part is as an icon. A document part is a part whos? 
purpose is ultimately to be printed on paper or other hani copy, 
such as a text document or a graphics document. The preferred 
5 representation for a part that is contained in a document part is as a 
ftame. Thus, referring to the operation depicted in Figures 7A-7C, 
the graphic part is illustrated as an icon when it is present on the 
desktop (Figure 7A), but is changed to a fiame iq?resentation when 
it is placed within the document (Figure 7C). Similarly, when the 
10 part is moved from a document to the desktop, its representation 
changes from a frame to an icon (Figures 7C-7E). Of course, the 
user can be provided with the option to change the representation of 
any part, for example through a keyboard or menu command. 

As described previously, a stationery part is different in the 
15 sense that it is only represented as an icon. It can be moved from 
one desktop part to another. Thus, for example, it can be treated 
like an ordinary part in a folder. However, when an attempt is 
made to move a stationery icon into a document part, a copy of the 
stationery part is placed in the document part instead, and that copy 
20 appears as a frame. 
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In addition to moving parts as a whole, the user can select 
and move some or all of the contents of a part. Every part has its 
own intrinsic type of contents. For example, text parts contain text 
characters, graphic parts contain graphic elements, etc. The user 
can select some of the intrinsic contents of one part, the donor part, 
and move or copy them into a different part, the destination part. 
There are three possible scenarios in which this type of operation 
can occur: (1) the donor and destination parts both contain the 
same intrinsic type of data, (2) the donor and destination parts 
contain different types of data, but the destination part can 
accommodate the category of data being received from the donor 
part, and (3) the donor and destination parts are of different data 
types, and the destination part is not adapted to accommodate the 
category of material received from the donor part. Each of these 
three cases is discussed below, with reference to illustrated 
examples. 

In the first scenario, the donor and destination parts are the 
same type of part. For example, their intrinsic content may be text 
characters, and they may have been created with the same word 
processor. Figure 9A illustrates an example in which two text 
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parts, namely documents 90 and 92, are respectively located within 
two windows on a desktop. A portion 94 of the text in the left- 
hand document 90 has been selected, as indicated by the reverse 
video display. This selected text is to be copied into the document 
92 in the right-hand window. Thus, the user dq)iesses the mouse 
button, along with a function key, if required, and drags the copy 
95 of the selected text 94 into the right-hand document 92, as 
shown in Figure 9B. Once the cursor eatNs the second documait 
92, this document becomes highlighted. Since the left-hand 
document 90 is now inactive, it is placed behind the document 92, 
and its title bar is no longer highlighted. 

Once the copied text is in the desired location, the user 
releases the mouse button to place the text in the document, as 
illustrated in Figure 9C. Since the text that was copied from the 
first document to the second document is of the same type as the 
intrinsic contents of the second document, that text is incorporated 
into the contents of the second document. In other words, once the 
selected text is placed in the second document, it no longer has a 
separate identity of its own. Rather, it becomes a part of the 
intrinsic contents of the destination document. Thus, it can be 
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edited with the same editor that is used for the original contents of 
the destination document. 

In the second scenario described above, the intrinsic 
contents of the donor and destination parts are of different types, 
but the destination part can handle the category of information 
contained in the donor part. Referring to Figure lOA, selected text 
96 from a document 90 created with a full capability text editor is 
to be copied into a dialog box 98. In this example, the selected 
text 96 is stylized, i.e. it has a bold attribute and is in an italicized 
font. The dialog box, however, is only adapted to handle plain 
text. 

In a manner similar to the previous example, a copy of the 
selected text 96 is dragged from the text document to the dialog 
box. As the cursor enters the dialog box 98, it moves to the 
forefront of the display, as illustrated in Figure lOB. Once the 
copy of the selected text is located in the appropriate portion of the 
dialog box, the user releases the mouse button to "drop" it in place. 
At this point, the system determines that the dropped text is not of 
the same type as the intrinsic contents of the dialog box, i.e. it is 
stylized rather than plain. However, it belongs to a compatible 
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category, in that it is text. Accordingly, the system converts the 
stylized text into plain text by removing its style properties, and 
then incorporates it into the dialog box as part of its intrinsic 
contents, as shown in Figure IOC. Note also that the copied text, 
which was wrapped over two lines in the original document 90, has 
been rewrapped onto a single line in a dialog box 98 as part of the 
conversion process, for example by removing a carriage return in 
the copy of the selected text. 

In the third scenario described above, the intrinsic contents 
of the donor document are of a different type and an incompatible 
category, relative to those of the destination document. Referring 
to Figure llA, selected text 94 from a text document 90 is to be 
copied into a graphic document 100. Again, the user drags a copy 
95 of the selected text from the text document 90 to the graphic 
document 100, which is highlighted as soon as the cursor enters the 
destination document (Figure 1 IB). Once the copy 95 of the text is 
in the appropriate location, the user releases the mouse button to 
drop the text into the graphic document. In this case, the category 
of the selected material, i.e. text, is not compatible with the 
intrinsic contents of the destination document, namely graphic 



WOM^7228 



PCT/US94/05027 



44 

elements. Therefore, the system creates a new part of the same 
type (text) as the donor document, inserts the copy of the selected 
material in it, and embeds the new part in the destination 
document. With this approach, the new part retains its own 
5 identity within the destination document. It has its own intrinsic 
content, i.e. text characters, and a text editor associated with it. 
Since the new material is a separate part, it resides within the 
graphic document as a frame 102, as illustrated in Figure 11 C. 
The frame 102 remains as the selected object, as illustrated by the 

10 border around the frame. 

As the copy of the selected text was being moved from the 
text document to the graphics document, the menu bar 104 of the 
desktop retained the commands associated with a text document, as 
illustrated in Figures llA and UB. However, once the copied text 

IS was dropped into the graphics document, the commands available 
on the menu bar 104 switched from those appropriate for a text 
editor to those for a graphics editor, as illustrated in Figure IIC. 
In addition, a tool bar 106 associated with the graphics editor 
appeared on the desktop, since the graphics document is now the 

20 active document in which the selected part is located. If, however, 
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the user positions the cursor within the frame 102 and selects a 
character or a word, the tool bar 106 is removed and the commands 
in the menu bar 104 return to those appropriate for editing text. 
In summary, when contents are moved from one document 
5 to another, the system determines whether the contents can be 
incorporated into the destination document as part of its intrinsic 
contents, or must be embedded in the destination document as an 
autonomous part. The operation of the system to carry out this 
process is illustrated in the flow chart of Figure 12. After the user 

10 has selected an object and dragged it to its intended destination, the 
system determines the type of part from which the selected object 
originated (Step 1201). For example, the selected object might be 
plain text, styled text, a graphic element, or the like. One of the 
properti^ that is stored with each part identifies which type of part 

15 it is, and each object which is moved or copied from the part is 
also identified as being of that type. After the part's type is 
identified, a determination is then made whether this type is the 
same as that of the destination document (Step 1202). If it is, the 
selected contents are incorporated or inserted into the destination 

20 document, to become part of its intrinsic content (Step 1203). If 
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the selected infonnation is not of the same type as the destination 
document, the system determines whether they belong to the same 
category (Step 1204). For example, if the selected information is 
styled text and the destination is plain text, they belong to the same 
S category, although they are different types. If the selected 
information belongs to the same category as the destination 
document, the system converts the selected information into the 
same type as the destination document (Step I20S). For example, 
text that is generated by one word processor can be converted into 

10 the format employed by a different word processor, or graphic 
documents can be converted from one data format to another. 
Once the conversion is complete, the selected information is 
incorporated into the contents of the destination document (Step 
1203). If the selected information is not of the same category as 

IS the destination document, a new part is created with the selected 
information as its contents (Step 1206), and this new part is 
embedded in the destination document as a frame (Step 1207). 

Because of their autonomous nature, parts greatly facilitate 
the exchange of information between various users of a system. A 

20 document, i.e. a root part, can contain several other parts. A user 
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can retrieve the entire document to view it and edit the individual 
parts as desired. Alternatively, the respective parts can be 
individually placed on a desktop that is shared by many users, 
while access to the document as a whole is protected. 
5 Further along these lines, the individual parts can be 

protected with limited access rights. For example, a document 
containing several different parts may be intended for distribution 
to a number of different categories of people. Some of the parts 
may contain confidential information which is not to be revealed to 

10 some of the categories of people. In the context of the present 
invention, these individual parts can be protected with limited 
access rights, and their contents can be viewed only by those who 
have an appropriate password, or other form of access. In other 
words, access restrictability is another characteristic property of a 

15 part. 

Referring to Figure 13, a document 108 is illustrated which 
is comprised of a number of parts, each of which is contained 
within a frame designated by dashed lines. One of the parts 110 is 
access protected. For example, this part may contain confidential 
20 spreadsheet data. Since the access to this part is limited, its 
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contents are not displayed within the frame. Rather, the frame is 
opaque, and can include an appropriate symbol, such as a key 112, 
to indicate the fact that a password or the like is required to gain 
access to it. 

5 The document 108 can be opened by users of the system, to 

view its general layout and the information in non-confidential 
parts. However, the data contained within the confidential part 110 
can only be seen by those who have an appropriate password for 
accessing the part. When such a penon selects the part, its opaque 

10 cover is removed and its contents are displayed. 

Although its contents cannot be accessed, the other 
properties of a restricted part are still available to the users of the 
system who do not have the password. For example, the location 
of the part within the document can be changed, and it can be 

IS resized by selecting it and moving the resize handles on the border 
of its frame. Likewise, the limited access part can be copied for 
placement elsewhere. However, the access protection remains with 
the copy of the part wherever it is located. 

A part with limited access rights can have other parts 

20 embedded in it. Consequently, if a user does not possess the 
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appropriate password to obtain access to the contents of a part, the 
other parts contained within it will also be restricted from view. 
Once a password has been accepted, the password can be 
automatically passed to all of the embedded parts so that they are 
S unlocked and viewable without further user involvement. Of 
course, however, the embedded parts could have different 
passwords associated with them, so that even if a viewer can obtain 
access to one restricted part, access to other parts within it may be 
further limited. 

10 Thus, the present invention can limit access to the contents 

of a document on an intra-document level. The access rights are 
part-specific, and effective regardless of where the data in the part 
is located. This approach permits non-privileged parts of a 
document to be accessed by users without imposing document-wide 

IS access restrictions. In large documents having multiple authors, the 
ability to limit access on a part-by-part basis provides much more 
flexibility in the viewing of the document and collaboration by the 
authors. 

To facilitate the manipulation of parts and compilation of 
20 compound documents, a special type of part, called a "parts bin". 
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can be provided. An example of a parts bin is shown in Figure 14. 
Referring thereto, a document is open in a window 120, and a parts 
bin 122 for the document appears in a separate area that is referred 
to as a windoid. The windoid is associated with the window 120, 

5 so that if the document window is brought to the front of the 
screen, the windoid comes along with it. 

The parts bin 122 is essentially a user-defined palette which 
functions as a source for other parts. It contains only other parts 
and has no intrinsic contents. The parts inside a parts bin have the 

10 property of being "frozen". More particularly, when a user 

normally depresses the mouse button while the cursor is located 
within a frame, the smallest element at the location of the cursor is 
selected. Referring to Figure 14, in the normal case, if the cursor 
is placed on one cell of a spreadsheet frame 124 and the cursor 

15 control button is depressed, the contents of that cell would be 

selected. The user may have intended to select a different object, 
e.g. the entire spreadsheet which contains the cell that was actually 
selected. To do so, the user must move the cursor to another area 
within the frame, but outside of other parts embedded within the 

20 desired part. This requirement is eliminated by freezing parts in a 
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parts bin, however. In this case, the user can click the mouse with 
the cursor located anywhere on a part and select the entire part. 
Thus, parts located within parts bins are dealt with as a whole, 
even if they are represented as frames which contain other 
S embedded parts. 

The parts in a parts bin can also be locked so that their 
contents cannot be modified. The locked property of the parts bin 
can be denoted by a padlock symbol 126 in the windoid's header. 
If a user attempts to drag a part out of a locked parts bin, a copy of 

10 the part is automatically made, and the original part remains in the 
parts bin, thereby ensuring the integrity of the bin's contents. To 
enable users to customize their parts bins, the lock on a parts bin 
can be turned off, for example by clicking on the padlock symbol 
126, and the parts modified. 

15 In a further embodiment of this aspect of the invention, 

every document can have an associated parts bin. A "document 
parts bin" enables a standard set of parts to be bundled with a 
document, so that users of that document will have those parts 
readily available to them. By activating a command, the user can 

20 cause the document parts bin to be displayed in a windoid or a 
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drawer. A document with an associated parts bin can have a . 
restrictive property placed on it, so that the only parts which can be 
placed in the document are those within the bin. An attempt to 
place other parts into the restricted document is rejected. This 
5 property ensures that users of the document work only with a 
certain set of parts, thereby reducing the training, support and 
maintenance associated with the document. For example, the parts 
bin for a data entry form will have only the parts necessary to enter 
data in the form. As a result, untrained users will not be exposed 

10 to unexpected parts, which could otherwise be used improperly. 
The restriction on parts also allows an organization to control the 
contents of documents. 

In addition to the user benefits described above, the 
architecture of the present invention provides benefits to the 

15 software developer. For example, a spreadsheet program is 

primarily designed to process numbers, and a developer who has 
expertise in this area can most efficiently spend time on this 
function. However, a text editor is a necessary part of a 
spreadsheet application, since the users must be able to edit the text 

20 in formulas and values. Having to write a text editor is a 



wo 94/27228 



PCTAJS94/05027 



53 

distraction for the developer of spreadsheet programs, although 
necessary with the conventional programs. However, in an 
arrangement in which a user-selected text editor is available for use 
as a separate part in a spreadsheet, it is no longer necessary for the 
S spreadsheet developer to write a text editor. The developer's time s 
can be devoted to the essence of the spreadsheet program, enabling 
extra features to be implemented, which may not be otherwise 
available because of time previously devoted to developing other 
features that were incidental to the number processing function of 

10 the spreadsheet. 

In addition, when the software developer decides to 
subsequently add new features or correct existing bugs, the task is 
made much easier. In particular, the developer need only 
concentrate on the spreadsheet aspects of the program, and need not 

15 be concerned with incidental components, such as a text editor. 

It will be appreciated that the present invention is not 
limited to the specific embodiments which have been described 
herein to facilitate an understanding of its underlying principles. 
For example, text and graphics have been used in the illustrative 

20 examples to represent the components of a document. The contents 
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of a part are not limited to these two specific categories, however. 
Rather, the contents of a part could be any type of media, including 
video, sound, animation, and the like. The scope of the invention, 
therefore, is defined by the claims which are appended hereto, 
S rather than the foregoing description, and all equivalents which are 
consistent with the meaning of the claims are intended to be 
embraced therein. 
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What is Claimed is; 

1. A system for controlling the addition of new material 
to a document stored in the memory of a computer having a 
display, comprising: 
S means for selecting new material to be added to the 

contents of a document and displaying said new material on said 
display; 

first means which identifies the new material as 
belonging to one of a plurality of categories of material; 
10 second means for identifying the intrinsic contents of 

the document as belonging to one of said categories; 

means for determining whether the category of the 
new material is the same as the category of the document's intrinsic 
contents; and 

15 means responsive to said determining means for (a) 

causing the new material to become incorporated as part of the 
intrinsic contents of the document and displaying the new material 
as part of the document's intrinsic contents, when the new material 
belongs to the same category as the document's intrinsic contents, 
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and for (b) establishing the new material as an object, embedding 
the object as a separate element in the document and displaying the 
document with the embedded object when the new material*s 
category is not the same as that of the document's intrinsic 
S contents. 

2. The system of claim 1 wherein said first identifying 
means indicates a data format within the category of material to 
which said new material belongs, and said second identifying means 
indicates a data format within the category of material to which the 

10 document's contents belong, and wherein said determining means 
determines whether the new material is of the same data format as 
the document's contents. 

3. The system of claim 2 further including means 
responsive to said determining means for converting the data format 

15 of the new material into the data format of the document's contents 
when the new material and the document belong to the same 
category and have different data formats. 
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4. The system of claim 1 wherein said system associates 
one or more predefmed properties with all data stored in a memory 
of the computer and stores those properties along with the data, 
with one of said properties being the category of material to which 

5 the data belongs, and wherein said first and second identifying 
means comprise means for reading the category property stored 
with said new material and the contents of said document, 
respectively. 

5. The system of claim 1 wherein said selecting means 
10 includes a pointing device which enables a user to identify the 

selected material, and means for displaying the selected material in 
a manner which distinguishes it from adjacent material. 

6. A method for controlling a computer to add new 
material to the contents of a document stored in a memory of the 

15 computer, comprising the steps of: 

classifying data stored in the computer as belonging 
to one of a set of categories; 
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determining within the computer whether data in the 
new material belongs to the same category as the existing contents 
of the document; 

automatically incorporating the data in the new 
S material as part of the data in the existing contents of the document 
and displaying the document with the incorporated material when 
the new material and the document are determined to belong to the 
same category; and 

automatically establishing the new material as an 
10 object, embedding said object as a discrete element in the 

document, and displaying the document with the embedded object 
when the new material and the document do not belong to the same 
category. 



7. The method of claim 6 further including the steps of: 
IS identifying the respective formats of the data in each 

of the new material and the document; 

determining whether the format of the data in the 
new material is the same as the format of data in the existing 
contents of the document; and 
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converting the data in the new material into the . 
format of the data in the existing contents of the document when 
the new material and the document are determined to belong to the 
same category and have different data formats. 



wo 94/27228 



PCT/US94/05027 



1/21 



CO 
CM 



CO 

a 



I 
I 
I 



o 

CNJ 



ID 
O 



O 

DC 



00 



CsJ 
CO 



V 



< 



I 

I 
I 



ARD 






o 






m 






KEY 




88 



CO 



5 

CO 
Q 



LU 



a. 



CM 



CO 
CM 



O 
CO 



wo 94/27228 



PCT/US94/05027 



2/21 



H File Edit Uietu Insert Forroat Font 



r 



38 



(?) » 



12 Hems (72MB in dM 164.7HB 



/'46 
Finances Report 



44^ 



45 



0^" 

Report 

Report 2 Blwkbrrriis 

,48 rr-K 44 




Glides Gtstioner^ U^ler 



I Text Document i 



This is a text document, I is a ted pat 
lhal has been opened into awhdow .When 
anoiher part is dragged into \ the part 
will open as a frame. 



50 



2 



40 



T 



34 



36 



/ 



42 



FIG 2A 



38 /50 



il File Edit Uieui Insert Format Font 




12 Items 672MB in disk 164.7MB 



'46 



Report 2 




44 



48 



■34 



SUdes Staticmerg Utter 



40' 



ID! 



I Text Document 



This is a te>a document, I is ate»dr p«t 
lhat hasbeen cpened into awhdow .When 
another part isdreioged into I the part 
will open as a frame. 




36 



42 



FIG 2B 



wo 94/27228 



PCTAJS94/0S027 



3/21 



58 



# File Edit Insert Format Font ^ 



56 



7 



iText Docj 



vcjsvsjvsvvccv c djsh 
ndn jsflsjsbj 
fy/er}\/ r jwj- 
Iwfjw vjrvj 
wrjwlfjw wp 
p4ir 4rip ej9 r 

wrjwlfjw wp wurpw 
p4ir 4ripejg ritegje rt 
9pk gej9 Idjg gld Isljj Ir 
j 1W jj le jut Iwrpjgjg kf 



FIG. 3A 



vcjsvsjvsvvccv c djsh 
sflsjslsjflsjkrk 
w rjwjflwj 
wjrwjfw wurpw 
f jw wp wurpw 
Inpejg ritegje rt 
jg Idjg gid Isljj Ir 
wrjwlfjw wp wurpw 
p4ir 4ripe jg ritegje rt 
9pk ge jg M jg gld Isljj Ir 
j lldjj lejut Iwrpjgjg kf 



54 



62 



60 ^68 



r / 





D 




O 


P 




□ 


o 


O 











V 



IText Doi 



YcjsvsjvsvvdCY c djsh 
ndn jsfb jslsj . 
fwer jw r jwj | 
Iwfjw wjrwj 
wrjwlfjw wp I 
p4ir 4ripejg r| 
9pk gejg Idjg gt 



wrjwlfjw wp wurpw 
p4ir 4ripejg ritegje rt 
9pk gejg Idjg gld Isljj Ir 
j lldjj lejut Iwrpjgjg kf 



/fcjsvsjvsYvccv c djsh 
,sf!sj$lsjfl$jkrk 
iw rjwjflwj 
wjrwjfw wurpw 
f jw wp wurpw 
i Jripejg ritegje rt 
jg Idjg gld Isljj Ir 



64 



FIG. 3B 



wrjwlfjw wp wurpw 
p4ir 4ripejg ritegje rt 
9pk gejg Idjg gld Isljj Ir 
j lldjj lejut Iwrpjgjg kf 



•54 



wo 94/27228 



PCT/US94/05027 



5/21 



FRAME: :ChangeFrameShape (shape) 



PART: :FrameShapeChanged (frame) 




FRAME: :ChangeClipShape (shape) 
or 

FRAME: :ResetClipShape () 



FIG. 5C 



wo 94/27228 



PCT/US94/05027 



6/21 




wo 94/27228 



PCT/US94/05027 



7/21 




wo 94/27228 



PCTAJS94/05027 



( BEGIN ) 



8/21 



FIG. 6 




NO 


610 




WARNING 




[RETURN ) 



wo 94/27228 



9/21 



PCTAJS94/05027 



H File Edit U'lew Uilml Special 




Blackberries 



TeKt Document 



This is a Text 
Document It has 
text in it. 



X 



80 



7 



FIG. 7A 



79 



# File Edit Uieui Insert Format Font (?) Q 



iP^= Tent Document ^=13= 




This is a Text 
Document. It has 
text in it 



Blackberries 



□ 



81 



80 



7 



FIG. 7B 



wo M/27228 



PCT/US94/05027 



10/21 

# File Edit Uiew Insert Format Font (?) ^ 



MJ^^m TeKt Document ^^I3§ 



TWsisaText 79 
Document It has — 
text in it 




H File Edit Uieui Insert Format Font (?) ^ 




wo 94/27228 



PCT/US94/05027 



11/21 



# File Edit Uiew Insert Format Font (J) Q 




TlusisaText 
Document It has 
text in it. 




80 



FIG. 7E 



wo 94/27228 



12/21 



PCTAJS94/05027 



USER ACTION 



USER POSITIONS MOUSE 
POINTER OVER ICON 
ON DESKTOP 



SYSTEM ACTION 



801 



USER PRESSES 
AND HOLDS DOWN 
MOUSE BUTTON 



802 



ICON IS HIGHLIGHTED 



USER MOVES MOUSE ' 
(WHILE MOUSE BUnON IS 
HELD DOWN, l.e. DRAGS) 



803 



804 




805 



FIG. 8A 



NO 



810 



. HIGHLIGHTING . 



USER XYES ^ 
RELEASES MOUSE>-— K^) 
BUnON? 



wo M/27228 



PCT/US94/05027 



13/21 




PLACE AND DISPLAY NEW PART 
AS ICON WITH UPPER-LEFT 
CORNER SAME AS CURRENT 
MOUSE POSITION 



PLACE AND DISPLAY NEW PART 
AS FRAME WITH UPPER-LEFT 
CORNER SAME AS CURRENT 
MOUSE POSITION 



812 




DONE 



FIG. 8B 



wo 94/27228 



PCTAJS94/05027 



14/21 



# File Edit View Insert Forraat Font 



ipi 



; TeHt Document 1 



This is Times, Bold 



italic text. 

look £ke if it s tnoved to 
ano titer paxt 94 



90 



lEIi 



ImiX DDi:umen1 2 



lectiott is simply incorporated 
s cofitefits of part B. For 
le, if they are both text parts, 
ected text is iriserud into the 
ation 's text at the point "where 
rdidis. 



FIG. 9A 



H File Edit View Insert Format Font 



77/15 is Times, Bold 



italic text. 

• ThisiAtistxates Yrhat itvoi^* 
kokEkeifitsroDvedtD 
another part 



ID! 



•♦•r fiy i*^* — 

n*fiis,is tunes, BoW] 



95 



TeHt Document 2 



The selection is simply incorporated 
into the contents of pejt B. For 
example, if they are both text parts, 
the selecud text is tnsertrd into the 
destination's text at the pointinrhere 
the user dicks. 



92 



FIG. 9B 



wo 94/27228 



PCT/US94/05027 



15/21 



^l^iieEditUieu^r^^ Font 



This is Times, Bold 
italic text. 

• This illustrates vh&t it vould 
look like if it is moved to 
another patt 



90 



TeHt Document 2 



Tills is TimesirBold 



The selection is simply incorporated 
into the contents of part B. For 
example, if they arehoth text parts, 
the selected text is insertrd into the 
destination's text at the point vhere 
the user clicks. 



S2 



FIG. 9C 



# File Edit Uietu Insert Format Font 



TeHt Document 1 



This is Times, 



italic text. 



♦ This illustrates vhat It >ffould 
look like if it is moved to 
another part 



90 



O 



96 



O 



el 



/ 




^3 



FIG. 10A 



wo 94/27228 



16/21 



PCTAJS94/05027 



it File Edit Uiew Insert Format Font ID # 





96 


This is Times,JljBIl!i 






« Thi£illttstt&tesNrhatitvottld 
look like if it is moved to 
enotlicr part 








Find: , ^ ^ 




(More Choices 1 [ Cancel ] ( Find ]] 















FIG. 10B 



# File Ed» Mem insert Format Font (?) ^ 







This is Times, Bold 
italic text. 

• This illtistr&tes^hat itvould 
look like if it is moved to 
another part 










■Smfl Bold, Italic teHt ■ 




(More Choices] [ Cancel ] [[ Find ]| 















FIG. IOC 



wo 94/27228 



PCT/US94/05027 



17/21 



1-104 



^ File Edit Uieui Insert Format Font 



iPi 



[TeHt Document I 



This is Times, Bold 



italic text. 

• This jAisstiatES ^vist itvoiM 
bok&Iceifits moved to ^ 
aiiothetpait ^ 




T' hcbkckbcrry is one of severail voody 
plants of the genus Ri^bus, having 
I candilce, ttsually thorny stems and blacky 



FIG. 11 A 



104 



# File Edit Uiew Insert Format Font (?) 



This is Times, 



italic text. 

• This illustrates wh 
look like if it is mov 
another part 



j2 



Graphics Document I 



1131 



\th is isjimes, B oTd "^ 
-1 1 ^ 



95 




100 




he blackberry is one of several voody 
{liants of the genus Rubus, having 
canelike, usually thorny stems and black. 



l.>VV^lf»« ^«K.I«. *^1fM-WV« 



FIG. 11 B 



wo 94/27Z28 



PCT/US94/05027 



18/21 



104 



^ File Edit Selection Brush Color 



um imnmmi \ 



This is Times, 
italic text. 
• Thisilltistiaiesvh 
loolcKkeifitstnoY 



ID! 







p 


\ 














\ 




m 





i Graphics Document 1 



PI 



yj/tw i s Times, Boldg 



7 



102 
106 





he l)lac]cberry is one of several voody 
_ plants of the genus Rubttfi^hanritig 
canelike, ttsiially thotny stems andhlack, 

wlftnmi a i ftU l^nwifftnn 



FIG. lie 



wo 94/27228 



PCTAJSM/0a»7 



19/21 



START 



READ 
SELECTED 
PART TYPE 



1201 




wo 94/27228 



PCTAJS94/05027 



20/21 



1^ 



Documentl 



..Im.*...I...*.. I^.A.J.?A.,.i...A.»L.A.Ml...A..X?A^J..«*,J...A.>.t. 



i=ii=jiiam 



Here are the nurrtoers for project X. 



The show looks great. 
You might want to run 
this by some of our 
recent hires to see what 
they get from it and if 
we have things covered. 

Chris 



Date Expense 


1/1/88 




1/5/88 


435345345 


1/5/88 


3241232 


1/5/88 


65456456 


1/5/88 


1 

o— ■«! 


1/6/88 


1/5/88 


1 


1/5/88 


1/5/88 





108 



ijPage 1 g[ 




FIG. 13 



wo 94/27228 



PCTAJS94/0S027 



21/21 



,126 



( Button I 

□ 



icon 



text frame 



predifined 
picture 



1 



graphics 
frame 



















































s 


pre 


Bd3 


heet 







122 



124 




FIG. 14 



INTERNATIONAL SEARCH REPORT 



interr. lal Application No 

PCT/US 94/05027 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC 5 G06F15/20 G06F3/033 



Accorting to International Patent Oasnfication (IPQ or to both national clasaficalion and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (dassification system followed by classification symbols) 

IPC 5 G06F 



Documentation seaiched other than minimum documentation to the extent that such documents are included in the fields searched 



Blectrontc data base consulted during the international search (name of daU base and, where practical, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Caiegpry * Gtaiiaa of document, with indication, where appropriate, of the relevant passages 



IBM SYSTEMS JOURNAL., 

vol.31, no. 3, 1992, ARMONK, NEW YORK US 

pages 429 - 458 

BERRY R.E. 'The designer's model of the 

CUA Workplace' 

see page 434 - page 439 

IBM TECHNICAL DISCLOSURE BULLETIN., 
vol.27, no.lOA, March 1985, NEW YORK US 
page 5518 

'Implementing Implicit Drag Along Without 
Regard to Object Set Type* 

-/-- 



Relevant to claim No. 



1-7 



1-7 



Further documents are listed in the continuation of box C 



Patent family members are listed in annex. 



' Special categones of ciud documents : 

'A' document defming the general state of the art which is not 

conadcred to be of particular relevance 
'E' earlier document but published on or afler the international 

filing date 

X" document which may throw doubts on priority daiin($) or 
which is a ted to esublish the publication date of another 
atation or other speaal reason (&s speafied) 

'O* document refcmng to an oral disclosure, use, exhibition or 
other means 

'P' document published pnor to the intcmational filing date but 
later than the pnonty date claimed 



T" later document puUidied alter the international filing date 
or prionty date and not in conflict with the aj^ication but 
ated to understand the prinaple or theory underlying the 
invenoon 

'X* document of putictdar relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document » taken alone 

*Y* document of particular relevance; the claimed invention 
cannot be considered to involve an inventive step when the 
docimient is combined with one or matt other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

document member of the same patent family 



Dale of the actual completion of the international search 



7 September 1994 



Date of mailing of the international search report 



Name and mailing address of the ISA 

European Patent Office, P.B. SSI 8 Patentlaan 2 

NL-22S0HVRiiswi)k 

Td. ( + 31-70) 340-2040, Tjl 31 651 epo nl, 

Fax (1- 31-70) 340-3016 



Authorized officer 



Suendermann, R 



Foim PCT/ISA/310 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 


bittm >l Application No 

PCT/US 94/05027 


C^Continuatian) DOCUMENTS CONSIDERED TO BE RELEVANT 


Category* 


Gtation of document, with indicatjon, where appropnatc. of ihc relevant passages 1 


Lelevant to claim No* 


X 


CHI '92 CONFERENCE PROCEEDINGS, 7 May 1992, 
MONTEREY, CALIFORNIA, USA 
pages 267 - 272 

OLSEN D.R. ET AL. 'WORKSPACES: AN 
ARCHITECTURE FOR EDITING COLLECTIONS OF 
OBJECTS' 




1.6 


A 


W0,A,92 08199 (60 CORPORATION) 14 May 1992 
see the whole document 




1-7 


A 


US,A,5 142 618 (FUJIWARA ET AL.) 25 August 
1992 

see the whole document 




1.6 


A 


EP,A,0 173 125 (IBM) 5 March 1986 
see the whole document 




1,6 


A 


IBM TECHNICAL DISCLOSURE BULLETIN., 
vol.27, no. 9, February 1985, NEW YORK US 
pages 5049 - 5052 

'Common Editing Arrangement for Different 
Object Types in Text Processing' 




1»6 


A 


IBM TECHNICAL DISCLOSURE BULLETIN., 
vol.34, no.lOB, March 1992, NEW YORK US 
pages 202 - 203 

'DRAGGING MARKED DATA TO AN EDITOR WINDOW' 




1,6 


A 


US,A,5 140 677 (FLEMING ET AL.) 18 August 
1992 

see the whole document 




1.6 


A 


EP.A.O 530 122 (IBM) 3 March 1993 
see the whole document 




1,6 



Poim PCT/ISA/210 (Gontlmiattoti of ueoAd ibtet) (July 1993) 



page 2 of 2 



INTERNATIONAL SEARCH REPORT 

InfonnAtion on patent family manbcn 



Inteni al Application No 

PCT/US 94/05027 



Patent document 


Publication 


Patent fiunily 


Publication 


died in search report 


date 


inember(s) 


date 


WO-A-9208199 


14-05-92 


AU-A- 


9031191 


26-05-92 






CA-A- 


2093123 


01-05-92 






EP-A- 


0555403 


18-08-93 






JP-T- 


6502734 


24-03-94 


US-A-5142618 


25-08-92 


JP-A- 


63131265 


03-06-88 






JP-A- 


63138451 


10-06-88 






JP-A- 


63138452 


10-06-88 






JP-A- 


63138453 


1A AA 

10-06-88 






JP-A- 


63138454 


10-06-88 


EP-A-0173125 


05-03-86 


US-A- 


4723210 


02-02-88 






OE-A- 


3586790 


10-12-92 


US-A-5140677 


18-08-92 


NONE 






EP-A-0530122 


03-03-93 


NONE 







Fom PCT/ISA/aiO (pttcni ramOy uimk) (July 1992) 



